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PATH DECOMPOSITIONS FOR MARKOV CHAINS 

By Gotz Kersting and Kaya Memi§oglu 

University of Frankfurt 

We present two path decompositions of Markov chains (with gen- 
eral state space) by means of harmonic functions, which are dual to 
each other. They can be seen as a generalization of Williams' de- 
composition of a Brownian motion with drift. The results may be 
illustrated by a multitude of examples, but we confine ourselves to 
different types of random walks and the Polya urn. 

1. Introduction and main results. In his paper on path decomposition 
and local time for diffusions Williams has given an appealing decomposition 
of a Brownian motion with negative drift (say —1) at its global maximum 
M [Williams (1974) and Pitman (1975)]. Let us state it as follows. 

Theorem 1 (Decomposition of a Brownian motion). Let X = (Xt)t>o be 
a Brownian motion starting in with drift 1 and let M be an independent 
random variable with exponential distribution and expectation 1/2. Define 
the stopping time 

T := sup{t >0:X s <M for all s < t}. 

Moreover, let X = (Xt)t>o be a process starting in Xq = Xt{= M), whose 
conditional distribution, given X and M , is equal to that of a Brownian 
motion with drift —1, which is conditioned to stay below M . Then the process 
X = (X t )t> given by 

X t:= l*u t<T, 
l*t-r, t>T, 

is a Brownian motion with drift — 1. 
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This theorem has been the starting point for further investigations. Williams 
already generalized his result to one-dimensional diffusion processes, and 
Bertoin and Chaumont gave related paths decompositions for certain classes 
of Levy processes [Bertoin (1991, 1992, 1993) and Chaumont (1996)]. Mil- 
lar (1978, 1977), Jacobsen (1974) and Greenwood and Pitman (1980) dis- 
cussed path decompositions from a broader point of view. Other path decom- 
positions like Tanaka's construction arose from conditioned random walks 
[Tanaka (1989, 1990)]. However, these results do not seem to suggest a gen- 
eral pattern how to deduce path decompositions for other Markov processes. 

In this paper we introduce a general method of path decomposing Markov 
processes by means of positive harmonic functions, which covers most of 
the mentioned results and thus offers a framework. We restrict ourselves 
to Markov chains, that is, to the case of a discrete time parameter. The 
technically more involved case of continuous time will be treated elsewhere. 

Let P(x,dy) be a probability kernel on some state space (S,S). No (topo- 
logical) restrictions are required for the state space. In the sequel X = 
(X n ) n6 N signifies a Markov chain with transition kernel P and time pa- 
rameter n G No := {0, 1,2,3,... }. The corresponding probability measure is 
denoted by P x , where x as usual is the initial state of (X n ). 

Further, let 

h:S->R, 0</i<oo 
be a nonnegative harmonic function with respect to P, that is, 

h(x) = J P(x,dy)h(y) 

for all x 6 S. Recall that for any nonnegative harmonic function h we may 
define the /i-transformed kernel P , given by 

P h (x,dy) : =-±-P( Xl dy)h(y) 
h[x) 

for all x 6 S with h(x) > 0. P h is a probability kernel on the restricted state 
space 

S h :={xeS:h(x) > 0}. 

Furthermore, h gives rise to probability measures with x £ S h , given by 

(1) KHXu .-,X n ) = / l (x)" 1 E a; 0(X 1 , . . . , X n )h(X n ) 

for any measurable function (fi:S n — > M. As is well known, under the mea- 
sure P^ the process (X n ) is a Markov chain with transition kernel P h . 

We are ready now to state the first main result of this paper, which gives 
a pathwise construction of (X n ) on a richer probability space. To emphasize 
this, we denote the corresponding probability measure by IP. Later in the 
section on random walks we will explain how this result fits to Williams' 
decomposition theorem. 
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Theorem 2 (Decomposition of a Markov chain). Let h > be harmonic 
and o € S such that h(o) > 0. Let X = (X n ) be a Markov chain with transi- 
tion kernel P h and initial state o, defined on a probability space with proba- 
bility measure F, and let Y be an independent random variable with values 
in (h(o), oo) and distribution given by 

y 

such that Y~ l is uniformly distributed on (0,/i(o)~ 1 ). Define the random 
variable 

T := sup{n > : h(X m ) < h(X n ) < Y for all m < n}. 

Moreover, let X = (X n ) be a process starting in Xq = Xt, whose conditional 
distribution, given X and Y , is equal to that of a Markov chain with tran- 
sition kernel P, which is conditioned to stay inside {x 6 S:h(x) < 
Then the process X = (X n ) given by 

X n . = fXn, n<T, 
{ X n _ T , n>T, 

is a Markov chain with transition kernel P, that is, equal in distribution to 
(X n ) under the measure P D . 

Variable T is the moment, when h(X n ) attains its global maximum for the 
first time. There are cases when T may also take the value oo with positive 
probability (in Section 3 we discuss this possibility in more detail), then no 
global maximum exists. In this case Theorem 2 holds with X n = X n for all 
n — this simply means that no concatenation with a process X takes place 
in this case. 

Note that T, in general, is not a stopping time, neither for X nor for X: 
It is the last moment, when h{X n ) exceeds all previous values, before h{X n ) 
surpasses Y for the first time. 




Variable T, thus, may contain information about the future behavior of 
X. The notable fact is that T is a splitting time, for X as well as for X . Split- 
ting times for Markov processes have been introduced by Jacobsen (1974) 
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just in the context of path decompositions, they fulfil a generalized Markov 
property, which corresponds to the Markovian character of X after the mo- 
ment T. In contrast to the classic definition of a stopping time, a splitting 
time T allows a change of law of the process (XT+ n )neN, depending on the 
value of T and Xj<, as in our case. For details compare the original work of 
Jacobsen (1974). 

It was remarked by Doob and others that there is a kind of duality be- 
tween the kernels P and P h [see Chapter 12.4 in Dellacherie and Meyer 
(1988)]. This is also reflected in our context: The next result presents a path 
decomposition for a Markov chain X = (X n ), whose transitions now obey 
the /i-transformed kernel P h . This dual decomposition takes place at the 
minimum of (h(X n )). 

Theorem 3 (Dual decomposition). Let h > be harmonic (for the ker- 
nel P) and o 6 S such that h(o) > 0. Let X* = (X*) be a Markov chain with 
transition kernel P and initial state o and let U be an independent random 
variable, uniformly distributed in the interval (0,h(o)). Define the random 
variable 

T* := sup{n > : h{X* m ) > h{X*) > U for all m < n}. 

Moreover, let X* = (X*) be a process starting in Xq = X^ t , whose con- 
ditional distribution, given X* and U , is equal to that of a Markov chain 
with transition kernel P h , which is conditioned to stay inside {x 6 S : h(x) > 
h(X?p*)}- Then the process X* = (X*) given by 

X* = iK, n<T*, 
n ' \X*_ T , n>T*, 

is a Markov chain with transition kernel P h , that is, equal in distribution to 
(X n ) under the measure P^. 

Remark 1 [Exact sampling of sup n h(X n )]. s\ip n h(X n ) cannot be sim- 
ulated directly because this would require infinitely many values X n . Given 
that T can be determined by finitely many simulation steps, our theorem 
offers an alternative: sup n h(X n ) and h{Xx) are equal in distribution, and 
the theorem gives a recipe to sample the latter random variable exactly. 
In Section 3 we give an additional condition that ensures that T can be 
determined algorithmicly. 

Remark 2. The conditioned Markov chains X, respectively X* , can 
also be obtained as unconditioned Markov chains, as we shall explain in the 
next section. In general, they are /i-processes of X only, given the value of 
(Xt,Y), respectively, (X£»,£7) (see Lemma 5 for details). 
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Remark 3. If g = ch for some constant c > 0, then P and P 9 are equal. 
Correspondingly, our constructions are the same for g and h up to the scaling 
factor c. One would also expect that, in essence, the path decompositions 
remains unaltered, if h is replaced by h + c. This is not easy to see directly 
for the first construction. Here duality is helpful, the transition from h to 
h + c can be well understood for the second construction. For details we refer 
to the proof of Theorem 3. 

Remark 4 (Doob inequality) . As an immediate consequence of our de- 
composition, we obtain the inequality 



This is Doob's martingale inequality, applied to the martingale (h(X n )). 

The paper is organized as follows. In Section 2 we prove both theorems. 
In Section 3 the question is addressed, how to detect the values of T and T* 
algorithmicly. In Section 4 we apply the decompositions to different types 
of random walks. 

2. Proof of the main results. Let us first have a closer look at the condi- 
tioned chain described in Theorem 2. It can be obtained as a Markov chain 
with suitable transition kernel. Let us recall this well-known construction. 
For s > define 

q s (x) :=P x (fe(JT i )<aVieN ) 

and the stopping time 

a s :=mf{i >0:h(Xi) > s}. 
Lemma 4. If h(x) < s, then q s (x) > 0. 

Proof. Assume a s < oo a.s. Then Fatou's lemma and the martingale 
property of (h(X n )) lead to 

B x h(X as ) = B x lim /i(X CTsAn ) < liminf E x h(X asAn ) = h(x) < s. 

n — >oo n — >oo 

Because h(X ag ) > s a.s., this is a contradiction. Thus, P x (o"s = oo) = q s (x) > 
and the lemma is proved. □ 

Furthermore, for h(x) < s the Markovian character of the chain gives 
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Thus, q s is also a harmonic function for the kernel P restricted to the set 
S s := {x € 5 : h(x) < s} and again for each s we can define a ^-transformed 
kernel Q s of P on the respective set S s by setting 

Q s (x,dy) := —]—P(x,dy)q s (y) 

= P x (X 1 edy\h(X i )<sVieN ). 

It follows that 

P X (X 1 e dxi, . .. ,X n Edx n \ h(Xi) < s V» E N ) 
_ P^jgi) • • -P(a: n _i,dx n )g s (x n ) 

= Q s (x,dxi) • • -Q s (a; n _i,dx n ). 
Altogether we obtain the following result. 

Lemma 5. Under P x the process (X n ), conditioned to stay inside S s , 
s > 0, is a Markov chain with transition kernel Q s (x,dy). 

The main step of our proof, which is based on a change of measure type 
argument, is contained in the next lemma. 

Lemma 6. Let h be harmonic and < h(o) < h(x) < s. Let U be uni- 
formly distributed in (0, /i(o) _1 ) and independent of (X n ) (with respect to 
P%). Then 

1 T \ Ko 



n hf 1 r rr 1 \ h (°) , \ 

x WW Ww<oo} < - W) ) = W) qh[x){x) - 

Proof. Since (h(X n )) is a P^-martingale and a s a stopping time, 

Px(o- s <n)= B x —— -; a s < n = E x [ — — -; a s < n 

[note that h(X asAn ) > on the event {o~ s < n}]. By means of (1) we rewrite 
this equation as 

Px(ff s <n) = h{x)V h x (—± -; OS < n) 
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As n —¥ oo, 

(2) P x (a s <oo) = h(x)E 

and, consequently, 

Qh(x)(x) = 1 - P x (o- h (x) < oo 



1 



{(7 S <00} 



(3) 



Since fapC^) -1 , h{x)~ l < h{o)~ l , by assumption, 



Ho) 



-i 



{cr s <oo}> 



1 



-/W Hx) <oo}<U< ^ 



(X r , 



h(x as y 

h(o) Ko) 
h(x) h(X ah( J {•*(-)<»>■ 



Taking expectations, the claim follows from (2) and (3). □ 

Proof of Theorem 2. Define 

r := inf {i > : h(Xj) < h(Xi) V j > i) 

to be the moment, where (h(X n )) attains its global maximum for the first 
time. First, we prove 

(4) P ((X 1 ,...,X n )eB,T = m)=¥((X 1 ,...,X n )eB,T = m) 
for natural numbers < m < n. Since 
{{X 1 ,...,X n )£B,T = m} 

= {(X! , . . . , X n ) e B n B m , n , h(Xj) < h(X m ) V j > n} 

with 

B m ,n ■= {(xi,.. .,x n ):h(xi),.. .,h(x m -i) 

< h(x m ) > h(x m+ i), h(x n )}, 

it follows that 

P ((X 1 ,...,J n )GB,r = m) 

P{o, dxx) ■ ■ ■ P(x n -i,dx n )q h ^ Xm )(x n ). 

BnB m . 

n 
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On the other hand, since h(Xx) = h(Xo) > h(Xi),h(X2), 
{{X 1 ,...,X n )eB,T = m} 

= • • • j X m ,Xi, . . . , X n - m ) G B n B mj7l } 

n ({/i(X m ) < Y < h(X & J,a m < 00} U {h(X m ) <Y,a m = 00}), 
where the stopping times a m are defined as 

<r m := inf{z > m : /ipQ) > fo(X m )}. 

It follows that 

F((X 1 ,...,X n )eB,T = m) 

= I P h (o,dx 1 )---P h (x m - 1 ,dx m ) 
JBr\B m 

,71 

x Qh{x m ){x m ,dx m+ i) ■ ■ ■ Qh(x m )( x n-i,dx n ) 



'( — ^—I{* m <oc} <y< —i- 



X m — %m 



= / P(o,dxi) ■ ■ ■ P{x n - 1 ,dx n )—— i — j — - 

xP ^ ( 1 j r 1 < M 

*"U(^^) { ^-> <oo} < y - ^(x m )y- 

Now (4) follows in view of Lemma 6 because Y~ l is uniformly distributed. 
Next, we prove 

(5) P ((X 1 ,...,X n )£B,T>n)=F((X 1 ,...,X n )eB,T>n). 
From 

{{X 1 ,...,X n )eB,T>n} 
= {(X 1 ,...,X n )GB} 

n {h(Xi) > max(/i(Xi), . . . , h(X n )) for some i > re}, 

it follows that 

P ((X 1 ,...,X n )eB,r>n) 

= / P(o,da;i)-"P(a; n _i,ds n )P a . n (o- max(/l(a . 1 ) v .. i/l(a . n) ) < 00). 

J B 

On the other hand, 

{{X 1 ,...,X n )eB,T>n} 

= {(Xx,. ..,X n )eB}n {a n < 00, h(X Wn ) < Y} 
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with 

a n := 'mf{i > n : h(Xi) > max(/t(Xi), . . . , h(X n ))}. 
It follows that 

¥((X 1 ,...,X n )eB,T>n) 

= f P h {o 1 dx 1 )---P h {x n . ll dx n ) 

JB 

X n = 2 



1 1 



[ P(o, d Xl )---P(x r 

JB 



-i,dx r . 



h(x n ) 



h(o) 

yh 



Y ~ k(X ) {°'™^(h(x 1 ),...,h(xn)) <00 } 

V (T max(/i(i^),...,/i(i n ))/ 



Since Y 1 is uniformly distributed and because of Lemma 6, (5) also follows. 
Combining (4) and (5), we obtain 

P((A > 1 , . . . , x n ) eB) = P ((x u . . . , x n ) e B) 

and the theorem is proved. □ 

Theorem 3 could be proved along the same lines, with some marked dif- 
ferences. We prefer to deduce it from Theorem 2 via duality. 

Proof of Theorem 3. First assume that h(x) > for all x 6 S. Then, 
as is easily seen, l/h(x) is harmonic with respect to P h , moreover, the 
l//i-transformed kernel of P h is equal to P, (i 3 ' 1 ) 1 /' 1 = P. Also Y^ 1 is uni- 
formly distributed in (0, (l//i(o)) _1 ) = (0, h(o)). In this situation Theorem 
3 is an immediate consequence of Theorem 2. 

Next, let us assume that h(x) also takes the value 0. Then h(x) + e is a 
strictly positive harmonic function for e > with respect to P. The formula 



(6) P h+£ = P h + - P 

K) ° h{o)+e ° h(o)+e 



which follows from (1), enables us to tranfer the validity of the path de- 
composition according to Theorem 3 from h + e to h. On the one hand 
Po +e converges to P^, as follows from (6). On the other hand, the processes 
induced by our construction (resp. its components, the uniform random vari- 
ables, the splitting times, and the conditioned Markov chains) converge in 
distribution, as e — > 0. □ 
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3. Detecting the value of the splitting time. It is too much to expect that 
such a general decomposition result as ours will provide useful information 
for any Markov chain possessing nontrivial harmonic functions. There are 
examples with an exotic touch. 

As an attempt to distinguish favorable situations from less appealing 
cases, let us adopt an algorithmic point of view and ask the question: When 
is it sufficient to run the chain X for finitely many steps in order to detect 
the value of T? Recall that, in general, T is not a stopping time and cannot 
be observed immediately. Clearly T is determined by (X n ) n < Tc , where the 
stopping time r c > T denotes the moment of crossing the level Y, 

T c :=M{n>0:h(X n ) >Y}. 

Thus, on the event {sup ra h(X n ) > Y} = {t c < 00} only finitely many steps 
of X are required. 




The situation may become less desirable, however, if {Y > sup h} = {r c = 
00} occurs. Then it may happen that T has a finite value, which, neverthe- 
less, cannot be observed within finite time, neither by r c nor by some other 
mean. To put this in mathematical terms: We say that T is detected by a 
stopping time r (stopping time with respect to X; we allow that r depends 
on Y, too), if finite values of T may be recognized from finite values of r, 
namely, that the value of T is a.s. not bigger than r. Expressed in formulas, 
this means 

{T < 00} = {r < 00} = {T < t < 00}, P-a.s. 

Then T = sup{n < r : n < r c , h(X n ) > maxj <n h(Xi)}, P-a.s. Note that r c de- 
tects T, if {T < 00} = {t c < 00}, P-a.s. 

An instance where no T-detecting stopping time exists may be considered 
as unfavourable and we give an example at the end of this section (see 
Example 1). For a majority of harmonic functions this kind of phenomenon 
does not occur. We introduce now a class of harmonic functions leading to 
path decompositions with splitting times, which always can be detected by 
stopping times. 
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First observe that the condition 

(7) sup/i(X n ) = oo, P^-a.s., 

n 

for all x £ S h implies sup n h(X n ) = oo and r c < oo P-a.s. Then T is detected 
by r c . More generally, consider the condition 

(8) snph(X n ) = sup/i, P^-a.s., 

n 

for all x 6 £ , thus, sup n /i(X n ) = sup/i P-a.s. Then, in case that {r c = oo} 
occurs, T takes the value oo, if h(X n ) < sup/i for all n, and, otherwise, the 
smallest value n such that h(X n ) =suph. Thus, introducing the stopping 
time 

f := inf {n > : h(X n ) = sup h} 

T = f on the event {r c = oo}. It follows that T is detected by the stopping 
time r := r c A f . 

The most prominent class of functions fulfilling (8) is constituted by the 
minimal harmonic functions. Recall that a nonnegative harmonic function 
h is said to be minimal (or extremal), if for any other harmonic function k 
such that < k(x) < h(x) for all x £ S, it follows k = ch with some constant 

ce[0,l]. 

For a countable state space the following result is a consequence of Doob's 
theorem from Martin boundary theory. We give a proof for arbitrary state 
space. 

Proposition 7. If h is a minimal harmonic function, then 
h(X n ) —> svph, P^-a.s., 

for all xeS h . 
PROOF. Since 

/ P h (x,dy)h{yY 1 = [ h(x)- 1 P(x,dy)<h(xr 1 f P(x,dy)=h(x)-\ 
Js h Js h Js 

l/h(X n ) is a P^-supermartingale. Thus, from the martingale convergence 
theorem 

h(X n )^L, P^-a.s., 

where L is a random variable with values in (0,sup/i]. For 7 > define the 
function 

fc 7 (x):=P£(L< 7 ) 



12 G. KERSTING AND K. MEMI§OGLU 

for x £ S h . This function is harmonic with respect to the kernel P h because 
ky(x) = P%(\mLh(X n ) < 7 

= J sh P h (x,dy)P%(limh(X n ) <^ = J^P^dy)^). 
It follows that 

W)=/ P(x,« i9 ) ftW47W =/p(x, rf!/ ) ft ( 9 )fe y W, 

where k^(x) may be chosen arbitrary for x ^ S h . This last equation says 
that h • fey is harmonic with respect to P, also 

h- ky <h 



by definition of Since by assumption h is minimal, we conclude that 
and, therefore, 



h ■ kry = ah 



P h x {L< 1 ) = a 

for all x £ S h with some constant a = a 7 6 [0, 1]. 

Next we show that the only possible values for a are or 1. Consider the 
estimate 



P h J{L<i}n f]{h(x m )< 7 }\ 

\ m>n / 



<V h x {L<lMXn)<l)<V h x {L<l)- 

Since h(X n ) — ► L, P^-a.s., the left-hand side converges to P X (L < 7), as 
n — > 00, provided 7 is a point of continuity of L. Consequently, 

P£(fc(X n )< 7> L<7)->a. 
On the other hand, by the Markov property, 



P^(X n )< 7 ,L<7)= f P h Jh(X n )edy)P^L< 7 ) 

P h x (h(X n )edy)a 



L 



'y-Ky)<i 

■ h(y)<tf 
= aP h x (h(X n )< 7 ). 
If 7 is a point of continuity of the distribution of L, it follows that 
P h x (h(X n ) < 7, L < 7) - aP h x (L < 7) = a 2 . 
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Therefore, a = a 2 , respectively, a = or 1 for all 7 (up to countably many), 
which means that L has a degenerate distribution, 

L = {3 P^-a.s. 

for some constant (3 € (0, sup h] . Also (3 does not depend on the starting 
state x of (X n ). Since l/h(X n ) is a nonnegative P^-supermartingale, we 
conclude by Fatou's lemma, 

h(x)- 1 > limE^h(X n y l > E^L" 1 = /T 1 

or h(x) < [3 < sup/i for all x € S . Therefore, L = sup/i, P^-a.s., which is 
our claim. □ 

An interesting example with different harmonic functions is provided by 
Polya's urn scheme. We show now that some of these functions lead to path 
decompositions with detectable splitting times, while others do not. 

Example 1. In Polya's urn scheme each drawn ball is put back into the 
urn together with an extra ball of the same color (two different colors). We 
consider the Markov chain X n = (R n ,T n ), n > 0, with states x = (r,t) £ N 2 , 
< r < t, where r denotes the number of red balls and t the total number 
of balls in the urn. The transition probabilities are 

{r 

-, for x = (r,t), y = (r + l,t + l), 
t 
v 

1--, for x = (r,t), y= (r,i + l). 

Then for given < p < 1, q = 1 — p, 

h p { X ) :=(i-i)(^ 2 i y-y-'-- 1 

is a harmonic function. Here suph p = 00. A quick calculation shows Pxy =p, 
respectively, q, such that X is a random walk. It is not difficult to conclude 
that (7) is fulfilled. This also follows from the last proposition because it is 
well known that these harmonic functions are minimal [Blackwell and Kendall 
(1964)]. Consequently, the associated path decompositions possess detectable 
splitting times. (The corresponding dual decomposition establishes an exotic 
path decomposition of a random walk by means of a Polya urn.) 
Also, 




is a harmonic function. In this case P% y = j^j, respectively, 1 — con- 
sequently, under the measure the shifted random variables X n + (1, 1), 
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n > 0, can be viewed as a Polya urn (with one extra red ball added to the urn 
at the beginning). We like to show that the corresponding path decomposi- 
tion gives rise to an undetectable splitting time T. To this end we make use 
of the familiar facts that h(X n ) = R n /T n is a.s. convergent and that, given 
the limit L, the random variables Z n := R n — R n -i, n>l, are independent 
Bernoulli variables with success probability L. These statements are valid 
with respect to P x as well as P^. 

First, we show that T < oo, P-a.s. Write X n = (R n ,T n ), Z n = R n — Rn-\ 
and L = lim n h(X n ) F-a.s. Then from R n = Rq + J27=i %i an d T n = h + Tq, 
it follows 

|sup/i(l n ) < l| = jsup£(Zi ~L)< Lfo - i? |, 

thus, by the properties of sums of i.i.d. Bernoulli variables {sup n h(X n ) < L} 
has zero probability. In other words, the moment 

N := infjn >0:h(X n ) = sup/i(X m )|, 

when for the first time h(X n ) obtains its global maximum, is finite P-a.s., 
which implies T < oo, P-a.s. Furthermore, sup n h(X n ) < 1, such that {r c = 
co} = {sup n h(X n ) < Y} has strictly positive probability. Therefore, r c does 
not detect T. 

In fact, there is no stopping time detecting T at all. For any P-a.s. finite 
stopping time r we have {N < t} = {sup J>0 /t(X r+ j) < M}, P-a.s., where 

M := max n < T h(X n ). Thus, from the strong Markov property 

P(iV <t)=Ki/)(X t ,M) with ip(x,m) := P*( suph(Xj) < m] . 

\j>o J 

Since i/j(x, m) < 1 for m < 1 and since M < 1, P-a.s., it follows P(iV < r) < 1. 
Similarly, P(iV <r | Y) < 1, P-a.s. (recall that we allow r to depend on Y). 
Because T = N on the event {Y > 1}, we obtain 

p(y > i, r < r) = p(y > 1, iv < r) 

= E(P(JV < r|F); y > 1) < P(Y > 1) 
and, consequently, P(T <r) < 1. Since T < oo, P-a.s., r cannot detect T. 

Remark 5. The class of harmonic functions given by (8) can be de- 
scribed in different ways. Given x E S h , we have the following equivalent 
conditions: 

suph(X n ) = sup/i, P^-a.s. 
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]imh(X n ) = sup/i, P^-a.s. 
lim/i(X„) € {0,sup/i}, P x .-a.s. 

For the proof we refer to the following facts, which follow from the martingale 
property of (h{X n )) with respect to P x and the supermartingal property 
of (l/h(X n )) with respect to P^: (i) h{X n ) is a.s. convergent to a random 
variable L, with respect to both measures P x and P?J, where L < oo, P x -a.s. 
and L > 0, P^-a.s., (ii) if h(X n ) = sup/i for some n, then h(X m ) = sup/i, P^- 
a.s. for all m>n, (iii) the measures P^ and P x are related by the formulas 

dP£(- n {L < oo}) = ^ dP„(.), dP,(- n {L > o}) = M dph ( .) 

[see Theorem 1, Section VII. 6, in Shiryaev (1995)]. In much the same manner 
it follows that under the assumption h(x) < sup/i, 



inf h(X n ) = inf h, P x -a.s. 

n 

\mxh(X n ) = inf h, P x -a.s. 

n 

\imh(X n ) G {inf/i, oo}, P^-a.s. 

These conditions describe a class of harmonic functions such that the split- 
ting time T* of the dual path decomposition is detectable by some stopping 
time. Combining both sets of conditions, we obtain the following character- 
ization of (7): If x G Sfi, then 

suph(X n ) = oo, P^-a.s. <==> inf /i(X n ) = 0, P^-a.s. 

n n 

This can also be expressed in terms of the crossing time r c and its dual 
companion, 

t* :=inf{n>0:/i(X*) < U] 

as follows, 

r c < oo, P-a.s. -<=>■ t* < oo, P*-a.s. 



4. Examples: random walks. In this section we apply our theorems to 
random walks (X n ) in S = that is, Markov chains, whose transition 
kernels are shift-invariant, 

P{x,dy)=P(0,-x + dy) Vx,yGM d . 

We assume that the kernel is nondegenerate. 
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4.1. Random walk with drift. First we consider the situation that the 
random walk has a drift, which means that the increments have finite ex- 
pectation unequal to zero, 

(9) ji := B Xi = J x P(0, dx) + 0. 
Furthermore, we assume that 

(10) (p(u) := [ P(0,dx)e M < oo 



for all u € M d , where (-, •) denotes the ordinary dot product in M. d . Note that 
fj, = grady?(0). 

Then, as it is easy to see, the functions 

h u (x) :=e {u ' x \ u£C, 

are harmonic, where 

C:={u:<p(u) = l}. 

The /i^-transformed kernels are given by 

P hu (x,dy) = P(x,dy)e {u ' y - x) . 

Note that these kernels are again shift-invariant, which means that (X n ) is 
a random walk also with respect to the measures P^ u . The corresponding 
drift is 

li u :=E% u X 1 = J xP h "(0,dx)= J xe {u ' x) P{0,dx) = grady>(u). 

It is well known that C is a smooth manyfold in M. d of codimension 1. 
For us the following elementary property is of interest: For u £ C, u ^ 0, 
let y> u (A) '■= <£>(A • u) = Eoe x ( u ' Xl \ AeK, the moment generating function of 
(u,Xi). By assumption this random variable is nondegenerate, therefore, <p u 
is strictly convex. Furthermore, y u (A) = 1 for exactly two values, namely, 
and 1, therefore, ^4(0) < and (p' u (l) > 0. By means of the chain rule, 
these derivatives are easily calculated as (ti,grad<£>(0)) and (u,gradip(u)), 
such that 

(n,/u)<0, {u,fj, u )>0 for all u G C, u ^ 0. 

Applying the law of large numbers to the random walk (X n ), we obtain 

h u (X n )^0, P^-a.s., and h u (X n ) — > oo, P£ u -a.s. 

Thus, (8) is fulfilled. Coming to path decompositions, we obtain that the 
corresponding splitting and crossing times are a.s. finite, 



T < r r < oo a.s. and T* < r * < oo 



a.s. 
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Let us describe our first path decomposition in more detail. We choose o := 
G R d , thus, h u (o) = 1. Let Y be as in theorem 2, that is, F(Y > y) = l/y 
for y > 1 , and let 

M := |H| _1 logY: 

Then P(M > m) = P(Y > e ll"ll m ) = e'^™, thus, M is exponentially dis- 
tributed with expectation . Moreover, 



h u {x) > Y 




x£H u (M), 



where H U (M) := {x : (u, x) > \\u\\M} is a halfspace in W 1 , which has distance 
M from the origin. Altogether we obtain for each m£C,u/0, a different 
recipy to simulate the original random walk with initial state 0. 

1. Let M be exponentially distributed with expectation ||it|| . Let X be 
an independent random walk with transition kernel P hu . 

2. Define 

T:=sup{n>0:X n ^H u (M),(u,X m ) < ( M ,i„)Vm<n}. 

This is the first moment when (X n ) is closest to the halfspace H U (M), 
but has not yet entered it. 

3. Let X be a random walk with transition kernel P and initial position 
Xq = Xt, conditioned not to leave the halfspace, and put 

X . = [X n , n<T, 
\x n - T , n>T. 

Then (X n ) is a random walk with transition kernel P. 

In particular, if (X n ) is a one-dimensional random walk with negative drift, 
then C contains (besides 0) just one positive number u. The /i-transformed 
process X is a random walk with positive drift, and T is the last moment of 
a record value, before X exceeds M. The corresponding path decomposition 
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is in complete analogy with Williams' path decomposition for Brownian 
motion with drift. 

The dual decomposition has a similar form. There are plenty of other 
harmonic functions for multi-dimensional random walks with drift, which 
lead to further (albeit less transparent) path decompositions. 

4.2. Driftless random walk with absorbtion. A random walk (Z n ) on the 
real line S = M with expected increments 



E Zi = J xP(0,dx)=0 



does not possess any interesting harmonic functions. Therefore we change 
the setting by making all negative states into absorbing states, that is, we 
consider a driftless random walk with absorbtion in (— oo,0), 

X n := Z rhn where r := inf{z > : Zi < 0}. 

Then P{x,dy) = P(0,—x + dy), if x > 0, and P(x,dy) = 5 x (dy), if x < 0, 
where 5 X is the Dirac-measure in x. It is well known [Bertoin and Doney 
(1994)] that 



h(x) :-- 




f(H k <x), x>0, 



x<0, 

defines a harmonic function with respect to P, where if/% are the strictly 
descending ladder epochs for the random walk (Z n ) with initial state 0. 

The process (X n ) also has under the measure P^, a clear meaning: It is 
equal in distribution to the process (Z n ), conditioned never to enter (— oo,0) 
[Bertoin and Doney (1994)]. 

Thus, our theorems offer two dual path decompositions: A path decompo- 
sition for driftless random walks with absorbtion in the negative numbers — at 
their maximal value, and a path decomposition for driftless random walks, 
conditioned to stay positive — at their minimal value. Details are left to the 
reader. 



4.3. Isotropic random walks on homogeneous trees. Consider a homoge- 
neous tree T r with r > 3 edges at each node. The tree can be seen as the 
Cayley-graph of a free group G generated by r elements A := {a\, . . . ,a r } 
with af = ■ • ■ = of = e, where e is the identity. Then each node x corresponds 
to exactly one formal string 

X — ai^di2 ' ' ' did 

with ij 7^ ij + i for all 1 < j < d— 1. Define the distance of a node x from the 
center e as 

\x\ := d if x corresponds to the reduced word 
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and let c(x,y) be the last common vertex in the graph for x,y £ T r . Two 
reduced nodes and y = a^ x cij 2 ■ ■ ■ aj l are neighbors if and 

only if \k — l\ = 1 and | ?/ | = 1, where y" 1 = a,, a 



ii^n-i ' ' ' a ji ■ 

Consider an isotropic nearest neighbor random walk on T r , that is, a 
random walk (X n ) with values in T r and 

P(x, xcLi) = — V 1 < i < r. 

Sawyer and Cartwright have shown [Sawyer (1997) and Cartwright and 
Sawyer (1991)] that in the isotropic case all minimal harmonic functions 
of this process are given by 

with ui = a^a^ ■ ■ ■ £ an infinite sequence corresponding to an endpoint at 
infinity of the tree T r . Thus, the /i^-transform corresponds to conditioning 
the isotropic random walk to end in w. 

In this case the constant harmonic function is not minimal, as it is given 

by 

JA m 

with \x being the uniform distribution on with mass one. This can be 
easily seen by a bare-hand calculation: 



1 v (r - n 2 i c ( x ' w )i-N 

^lxl-1 2- ^ L) 



r(r — l)l a 



\x\ 



r(r-l)k fc=Q 



\c(x,uj)\=k 



r(r 



^r }"-"'"' f -""'" 

k=0 
\x\-l 

+ ]T (r - 2)(r - l)!"!-*- 1 ^ - l) 2k ~\ x [ + (r - 1)M 

k=l k=l,.'., \x\-l k=\x\ 

= r(r _ 1) |,|- 1 ( 1 + (^- 1 ) N " 1 - 1 + ^- 1 ) N ) 
= 1. 
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Thus, the isotropic random walk can be constructed by first chosing uni- 
formly an endpoint uj and then starting a /^-transformed random walk. 

The transition kernel P hlJ of such a minimal harmonic function then 
is given by 

!r — 1 
, if du(x) < du(y), 
r i 
— — , dd u (x)>d u (y), 
r(r — 1) 

for adjacent nodes x, y, with the distance of x to uj defined as d LJ (x) := 
2\c(x,u>)\ — \x\. 

The rich harmonic function space allows both the direct decomposition 
and the dual decomposition. As in the case of the random walk, again we 
have 

inf hui(X n ) = 0, P-a.s. and sup/i w (X n ) = oo, P^-a.s., 

n n 

which means for the corresponding splitting times T and T*: 

T < t c < oo a.s. and T* < r * < oo a.s. 

Fix a harmonic function h u . For both decompositions we still need the 
kernels of the conditioned chains and, therefore, we need 

q s (x) ^AP^MXO^sVieNo), 

q*(x) := A*P^(MX 4 ) > sVi G No) 

with some A, A* > 0. The kernel Q s of a conditioned P-chain (resp. Q* of a 
conditioned P h chain) is then given by 

Q s (x,y) = l/q s (x)P(x,y)q s (y), Q* s (x,y) = l/q*(x)P h (x,y)q*(y). 

First we deduce a closed form for q s . The definition of q s immediately gives 
us 

q s (x) = P s ((r - l) d ^ x ^ < s Vi G N ). 

Note that both kernels P and P hu> , plus the harmonic functions /i^, only 
depend on d ul {x) = 2|c(x,w)| — and, therefore, the function q s also only 
depends on the value of d UJ (x). Thus, q s can be reexpressed as 



Qs(x) = q(du>{x) 



logs 



log(r - 1) 



with q:Z^ IR + defined as 

q(n) := AP((i w (X / ) < OVz G N | d u (X ) = n) 
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with A > and the boundary condition q(0) = 1. Furthermore, q has to 
satisfy the following recurrence equation as a direct consequence of the har- 
monicity of q s on the set {x : h(x) < s}: 

q(i) = -q{i + l) + 1 —q{i-l). 
r r 

Solving this recurrence with the two conditions q(0) = 1 and q(i) = for 
i > leads to the solution 

f (r-ir-r + 1 

?»= ^7 ' lfn ^ ' 

[ 0, else. 

Now we can construct the first decomposition of a free isotropic random 
walk on T r as follows (refer to the decomposition of a random walk for more 
details): 

1. Let M be exponentially distributed with expectation 1. Let X be an 
independent random walk on T r with transition kernel P htJ . 

2. Define 



T := sup{n > : d w {X n ) < M, d w {X m ) < d w {X n )\/m < n}. 

be a random wal 
tion Xq = Xt, and put 



3. Let X be a random walk with transition kernel Q h ,g T j and initial posi- 



. = (X n , n<T, 
\x n _ T , n>T. 

Then (X n ) is an isotropic random walk on T r . 

Some similar considerations in the case of a conditioned P huJ -process lead 
to the following recurrence equation for q*, 

rii) = r -^q h {i + l)+ l -q h {i-l) 
with the solution given by 



(r-1) 



The dual decomposition of a P huJ -chain first starts an isotropic random walk 
on T r and attaches a conditioned P huJ -chain with kernel Q*. Details are left 
to the reader. 
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